Illumination color covariant locale-based visual object retrieval

Search by Object Model — finding an object inside a target image — is a desirable and yet difficult mechanism for querying multimedia data. An added difficulty is that objects can be photographed under different lighting conditions. While human vision has color constancy, an invariant processing, presumably, here we seek only covariant processing and look to recover such lighting change. Making use of feature-consistent locales in an image we develop a scene partition by localization, rather than by image segmentation. A diagonal model for illumination change and a voting scheme in chromaticity space provide a candidate set of lighting change coefficients for covariant image transformation. For each pair of coefficients, Elastic Correlation, a form of correlation of locale colors, is performed along with a Least Squares minimization for pose estimation. Since the rotation, scale and translation parameters are thus estimated, we can apply an efficient process of texture support and shape verification. Tests on an image and video database of about 1,500 images show an average Recall and Precision of over 70%.